Content reproduction method and apparatus in IPTV terminal

ABSTRACT

An IPTV terminal is provided. It is possible to provide an additional service allowing a user to watch video content, in addition to a service for reproducing an audio file, by extracting metadata on the audio file if a reproduction command of the audio file is input, searching for the video content related to the audio file based on the extracted metadata, and receiving and reproducing the video content.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a continuation of U.S. application Ser. No.14/585,086, filed Dec. 29, 2014, which is a continuation of U.S.application Ser. No. 12/136,869, filed Jun. 11, 2008 (now U.S. Pat. No.8,924,417), in the United States Patent and Trademark Office, whichclaims priority from Korean Patent Application No. 10-2007-0092044,filed on Sep. 11, 2007, in the Korean Intellectual Property Office. Theentire disclosures of the prior applications are considered part of thedisclosure of the accompanying continuation application, and are herebyincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an Internet Protocol Television (IPTV),and more particularly, to a content reproduction method and apparatus inan IPTV terminal for reproducing an audio file and video content relatedthereto.

2. Description of the Related Art

An Internet Protocol Television (IPTV) provides an information service,moving picture content, and a broadcasting service by using theInternet. The IPTV is different from general internet television in thatthe IPTV uses a television instead of a computer monitor and is operatedby a remote control instead of a mouse.

FIG. 1 schematically illustrates a configuration of a general IPTVsystem.

Referring to FIG. 1, the IPTV system includes a television 110, an IPTVterminal 120, and an IPTV service provider server 130.

The television 110 serves to receive and reproduce audio and videosignals. The IPTV terminal (referred to as an ‘IPTV set-top box’) 120receives various contents through an internet line and reproduces thevarious contents in the television 110. The IPTV terminal 120 is locatedinside or outside of the television 110. The television 110 and the IPTVterminal 120 are connected to each other through a cable. The IPTVterminal 120 may be connected to the IPTV service provider server 130 orpublic internet network through the internet line.

A user can use various internet services such as a moving picturecontent service by connecting the IPTV terminal 120 to the television110. Accordingly, it is possible for a user who is not good at operatinga computer to use various contents and additional services such as amovie service, a home shopping service, a home banking service, onlinegames, and MP3 files provided through the Internet, in addition to asearch service provided through the internet, by using a remote control.

The IPTV is similar to a general apparatus for providing cablebroadcasting and satellite broadcasting contents in that the IPTVprovides broadcasting contents. However, the IPTV is different from thegeneral apparatus for providing cable broadcasting and satellitebroadcasting contents in that the IPTV has bidirectionality. Unlike ageneral apparatus for providing sky wave broadcasting or satellitebroadcasting contents, in the IPTV, it is possible for the user to watcha desired program at a desired time.

The IPTV provides functions such as an audio on demand (AOD) service, avideo on demand (VOD) service, a multicast service, a local music filereproduction service, and a browsing service. Accordingly, the user mayaccess the IPTV service provider server 130, receive the AOD service,and listen to music by using the IPTV. Alternatively, the user mayreproduce a music file previously downloaded into an internal storagedevice and listen to music (reproduction of a local music file).

However, a display device of the IPTV is not used to reproduce the VODor local music file in the IPTV terminal 120. Accordingly, it isdesirable to provide additional services by using the display device, inaddition to reproduction of an audio file by using the IPTV terminal120, if the user reproduces the audio file.

SUMMARY OF THE INVENTION

The present invention provides a content reproduction method andapparatus in an IPTV terminal for searching for video content related toan audio file, if reproducing the audio file.

In the present invention, it is possible to provide additional servicesby using a display device in addition to a reproduction service of anaudio file by extracting metadata on the audio file, searching for videocontent based on the metadata, and reproducing the video content.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings in which:

FIG. 1 schematically illustrates a configuration of a general IPTVsystem;

FIG. 2 is a functional block diagram illustrating a content reproductionapparatus in an IPTV terminal according a first exemplary embodiment ofthe present invention;

FIG. 3 is a functional block diagram illustrating a content reproductionapparatus in an IPTV terminal according to second to fourth exemplaryembodiments of the present invention;

FIG. 4 is a flowchart of a method of reproducing content in an IPTVterminal according to the first embodiment of the present invention;

FIG. 5 is a flowchart of an example of a method for synchronizing videocontent with an audio file according to the first embodiment of FIG. 4;

FIG. 6 is a flowchart of a method of reproducing content in an IPTVterminal according to the second embodiment of the present invention;

FIG. 7 is a flowchart of a method of reproducing content in an IPTVterminal according to the third embodiment of the present invention; and

FIG. 8 is a flowchart of a method of reproducing content in an IPTVterminal according to the fourth embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS OF THE INVENTION

The present invention will now be described more fully with reference tothe accompanying drawings, in which exemplary embodiments of theinvention are shown.

FIG. 2 is a functional block diagram illustrating a content reproductionapparatus in an IPTV terminal according a first embodiment of thepresent invention.

Referring to FIG. 2, a content reproduction apparatus 200 in the IPTVterminal includes a reproduction unit 210, an extraction unit 220, asearching unit 230, a receiving unit 240, and a synchronization unit250.

After receiving a reproduction command from a user, the reproductionunit 210 reproduces an audio file that has been previously downloadedinto a storage device (not shown) in the IPTV terminal or providedthrough an AOD service through a speaker of a television 110. Inaddition, the reproduction unit 210 receives video content from an IPTVservice provider server 280 or a public internet network 290 through thereceiving unit 240 and the synchronization unit 250 and reproduces thereceived video content through a screen of the television 110.

The extraction unit 220 extracts metadata on an audio file for which areproduction command has been input from a user. The extraction unit 220receives an audio file from the reproduction unit 210. The metadata onthe audio file may be extracted from the audio file. In addition, in amodified embodiment, metadata on a plurality of audio files stored inthe IPTV terminal may be managed as a database 260 in the IPTV terminalor stored and managed as a predetermined folder or file in a storagedevice (not shown) in the IPTV terminal. In this case, the extractionunit 220 may extract metadata from the database 260, the predeterminedfolder, or the predetermined file.

However, it is possible that the metadata may not be found in the audiofile, the database 260, the predetermined folder, or the predeterminedfile. In addition, even if the metadata is found, the metadata may notinclude sufficient information. In this case, the extraction unit 220may acquire all the extractable information on the audio file, access anaudio file provider or metadata provider 270, and receive additionalmetadata.

The searching unit 230 searches for video content related to the audiofile based on the metadata extracted by the extraction unit 220. Forexample, if the audio file is a music file, the video content relatedthereto may be a music video, a live broadcasting program of a singer,an original sound track video, or a broadcasting advertisement. Thevideo contents related to the audio file are searched for in the IPTVservice provider server 280 or public internet network 290.

In addition, if there are a plurality of video content items found inthe IPTV service provider server 280 or the public internet network 290,the searching unit 230 may select a type of video content received fromthe IPTV service provider server 280 or public internet network 290based on priority predetermined by the user. For example, if the foundvideo content is a music video or a live broadcasting program of asinger, the searching unit 230 may select a music video so as to allowthe music video to be transmitted to the receiving unit 240 based on thepriority of the user.

In addition, the searching unit 230 may select video content based onresolution thereof. For example, if there are a plurality of musicvideos related to the same audio files, the searching unit 230 mayselect a music video with high resolution so as allow the music videowith high resolution to be transmitted to the receiving unit 240.

The receiving unit 240 receives the video content selected by thesearching unit 230 from the IPTV service provider server 280 or publicinternet network 290. Although the receiving unit 240 is shown as afunction block separate from the searching unit 230 in FIG. 2, thereceiving unit 240 need not be separate from the searching unit 230. Insome embodiments, the receiving unit 240 may be included in thesearching unit 230. Accordingly, the searching unit 230 and thereceiving unit 240 need not be connected to the IPTV service providerserver 280 or the public internet network 290 through respectiveinternet lines. The searching unit 230 and the receiving unit 240 may beconnected to the IPTV service provider server 280 or the public internetnetwork 290 through a single internet line.

The receiving unit 240 receives the video content provided through theIPTV service provider server 280 or the public internet network 290 andtransmits the video content to the synchronization unit 250. Thereceiving unit 240 can receive the video content through a downloadingand storing process or a streaming process. For example, in a case wherethe streaming process is used, the video content received by thereceiving unit 240 is transmitted to the synchronization unit 250 afterbuffering the video content. In a case where the downloading and storingprocess is used, the video content downloaded into the receiving unit240 is stored in a storage device (not shown) in the IPTV terminal. Thevideo content for which a storing process has been completed istransmitted to the synchronization unit 250 through the receiving unit240.

The synchronization unit 250 synchronizes the video content received bythe receiving unit 240 with the audio file. The audio file may be inputinto the reproduction unit 210. For example, if the video content issearched for while the audio file is being reproduced, the video contentneeds to be reproduced from a start point found by jumping a periodequal to a period over which the audio file has been reproduced. Thesynchronization process may be performed in various manners. Forexample, it is possible to set a start time of the video content byusing the current reproduction time of the audio file. In addition, itis possible to set a start time of the video content by comparing pitchinformation of the audio file with pitch information of the videocontent. The video content synchronized with the audio file istransmitted to the reproduction unit 210.

The reproduction unit 210 reproduces the audio file and the videocontent synchronized with the audio file. The audio file and a videocomponent of the video content synchronized with the audio file may bereproduced. Alternatively, the reproduction of the audio file may bestopped, and the video and audio components of the video contentsynchronized with the audio file may be reproduced.

FIG. 3 is a functional block diagram illustrating a content reproductionapparatus in an IPTV terminal according to second to fourth embodimentsof the present invention.

Referring to FIG. 3, a content reproduction apparatus 300 in the IPTVterminal includes a reproduction unit 310, an extraction unit 320, asearching unit 330, and a receiving unit 340.

In the second embodiment of the present invention, the extraction unit320 extracts metadata on audio files stored in a storage device (notshown) in the IPTV terminal before a reproduction command is input froma user to the reproduction unit 310.

The searching unit 330 searches for video content in the IPTV serviceprovider server 370 or the public internet network 380 by using themetadata extracted by the extraction unit 320.

Since operations of the extraction unit 320, the searching unit 330, thedatabase 350, and a metadata provider or an audio file provider 360 aresimilar to operations of the extraction unit 220, the searching unit230, the database 260, and the metadata provider or audio file provider270 except that the metadata is extracted before the reproductioncommand is received from the user, detailed description on those will beomitted.

The receiving unit 340 receives link information of the video contentrelated to the metadata from the IPTV service provider server 370 or thepublic internet network 380. The link information of the received videocontent may be stored in a storage device (not shown) in the receivingunit 340 or stored in the receiving device in the IPTV terminal.

If receiving a reproduction command of an audio file, the reproductionunit 310 receives video content related to the link information throughthe receiving unit 340 and reproduces the video content in a streamingprocess.

In the third embodiment, since the extraction unit 320 and the searchingunit 330 operate similarly to those in the second embodiment, thedetailed description will be omitted.

The receiving unit 340 downloads and stores not only the linkinformation of the video content but the video content from the IPTVservice provider server 370 or the public internet network 380. Thedownloaded video content may be stored in the storage device in thereceiving unit 340 or stored in the storage device (not shown) in theIPTV terminal.

If receiving a reproduction command of an audio file, the reproductionunit 310 reproduces the video content through a downloading and storingprocess.

In the fourth embodiment, if a reproduction command of an audio file isinput into the reproduction unit 310 by a user, the extraction unit 320extracts metadata on the audio file. In the current embodiment, evenwhen a reproduction command is input by the user, the audio file is notimmediately reproduced. That is, after the video content is received,the video content is reproduced instead of the audio file.Alternatively, the audio file and a video component of the video contentare concurrently reproduced.

The searching unit 330 searches for video content related to the audiofile based on the extracted metadata.

The receiving unit 340 receives the video content from the IPTV serviceprovider server 370 or the public internet network 380.

The reproduction unit 310 reproduces the video content received by thereceiving unit 340 through a downloading and storing process or astreaming process.

FIG. 4 is a flowchart of a method of reproducing content in an IPTVterminal according to the first embodiment of the present invention.

Referring to FIG. 4, in operation 410, if a reproduction command of anaudio file is input by a user, the audio file is reproduced.

In operation 420, metadata on the audio file is extracted. The metadatamay be extracted from the audio file or a separate file from the audiofile. The separate file may be stored in the IPTV terminal as apredetermined folder or as a predetermined file so as to convenientlymanage a plurality of metadata on a plurality of audio files.

In operation 430, video content related to the audio file is searchedfor based on the metadata extracted in operation 420. The video contentmay be searched for in the IPTV service provider or the public internetnetwork through an internet line. The video content related to the audiofile includes, for example, a music video, a live broadcasting program,an original sound track video, or a broadcasting advertisement.

In addition, if there are a plurality of video contents found in theIPTV service provider server or the public internet network, only onevideo content may be received or reproduced based on prioritypredetermined by the user or based on resolution. For example, if thefound video content is a music video or a live broadcasting program of asinger, the music video may only be received and reproduced based on thepriority of the user.

In operation 440, if video content is found, the current processproceeds to operation 450. If video content is not found, the currentprocess proceeds to operation 480.

In operation 450, if video content is found in operation 430, the foundvideo content is received. The video content may be received andreproduced through a downloading and storing process or a streamingprocess.

In operation 460, the video content received in operation 450 issynchronized with the audio file. For example, if the video content issearched for while the audio file is being reproduced, the video contentneeds to be reproduced from a start point found by jumping a periodequal to a period over which the audio file has been reproduced. Acontent reproduction method in an IPTV terminal according to a modifiedembodiment may further include an operation of determining whether it ispossible to synchronize the video content with the audio file beforeoperation 460. For example, if a difference between the totalreproduction time of the audio file and the total reproduction time ofthe video content is very large, it may be determined that it isimpossible to synchronize the video content with the audio file. If itis impossible to synchronize the video content with the audio file, itis possible to proceed to operation 480.

In operation 470, a video component of the video content synchronizedwith the audio file is reproduced together with the audio file that isalready being reproduced. In a modified embodiment, the video componentof the video content may be reproduced not with the audio file but withan audio component of the video content.

In operation 480, if the video content is not found, at least one imagerelated to the audio file is searched. In a modified embodiment, apredetermined image stored in the IPTV terminal may be used. In thiscase, the current process is omitted.

In operation 490, the image searched in operation 480 is reproduced in aslide show manner.

FIG. 5 is a flowchart of an example of a method for synchronizing videocontent with an audio file in the first embodiment of FIG. 4.

Referring to FIG. 5, in operation 510, the total reproduction time ofthe audio file is compared with the total reproduction time of the videocontent.

In operation 520, if the total reproduction time of the audio file isthe same as the total reproduction time of the video content, thecurrent process proceeds to operation 530. Otherwise, the currentprocess proceeds to operation 540. In a modified embodiment, if it isdetermined that the total reproduction time of the audio file is notexactly the same as the total reproduction time of the video content butsubstantially the same as the total reproduction time of the videocontent, it is possible to proceed to operation 530.

In operation 530, if the total reproduction time of the audio file isthe same as the total reproduction time of the video content, thecurrent reproduction time of the video content is set so as to be thesame as the current reproduction time of the audio file. The videocomponent of the video content is reproduced starting from the currentreproduction time set in the current process.

In operation 540, if the total reproduction time of the audio file isdifferent from the total reproduction time of the video content, pitchinformation on a first time section, between a time that is apredetermined reference time earlier than the current reproduction timeand the current reproduction time of the audio file, is captured. Thesection in which the pitch information on the audio file is captured isnot limited to the aforementioned time section. Various changes may bemade. In addition, the predetermined reference time may be predeterminedby a user.

In operation 550, a second time section of the audio component of thevideo content having the same pitch information as the pitch informationon the first time section of the audio file is searched.

In operation 560, the current reproduction time of the video content isset based on the second time section and a time taken to search for thesecond time section. This is because time has elapsed while searchingfor the second time section in operation 550 and so the currentreproduction time of the audio file has increased. For example, inoperation 540, the first time section ranges from thirty seconds toforty seconds (that is, the current reproduction time is forty seconds,and the reference time is ten seconds). If it takes five seconds tosearch for the second time section, the current reproduction time of thevideo content is forty five seconds.

FIG. 6 is a flowchart of a method of reproducing content in an IPTVterminal according to the second embodiment of the present invention.

Referring to FIG. 6, in operation 610, metadata on an audio file isextracted before a reproduction command is input. The audio file may bepreviously stored in the IPTV terminal or provided by an audio fileprovider. The metadata may be extracted from the audio file or aseparate file from the audio file.

In operation 620, video content related to the audio file is searchedfor based on the extracted metadata.

In operation 630, if video content related to the audio file is found,the current process proceeds to operation 640. If video content relatedto the audio file is not found, the current process proceeds tooperation 660.

In operation 640, if video content has been found in operation 630, linkinformation of the video content is received. The received linkinformation is stored in the IPTV terminal.

In operation 650, if the reproduction command of the audio file isinput, the video content (video and audio components) is reproduced in astreaming process based on the link information of operation 640. Inthis case, the audio file is not reproduced.

In operation 660, if the video content has not been found, an imagerelated to the audio file is searched for and received. In a modifiedembodiment, if there is a predetermined image stored in the IPTVterminal, the current operation may be omitted.

In operation 670, if the reproduction command of the audio file isinput, the audio file is reproduced and the image in operation 660 isreproduced in a slide show manner at the same time.

FIG. 7 is a flowchart of a method of reproducing content in an IPTVterminal according to the third embodiment of the present invention.

Referring to FIG. 7, since operations 710 to 730 are similar tooperations 610 to 630 of FIG. 6, an additional description will beomitted.

In operation 740, if there is video content found in operation 730, thevideo content is downloaded and stored.

In operation 750, if the reproduction command of the audio file isinput, the video content (video and audio components) is reproducedthrough a downloading and storing process.

In operation 760, if the video content is not found, an image related tothe audio file is searched for and downloaded. If there is apredetermined image stored in the IPTV terminal, the current operationmay be omitted.

In operation 770, if a reproduction command of an audio file is input,the audio file is reproduced, and the image downloaded in operation 760is reproduced in a slide show manner, at the same time.

FIG. 8 is a flowchart of a method of reproducing content in an IPTVterminal according to the fourth embodiment of the present invention.

Referring to FIG. 8, in operation 810, if a reproduction command of anaudio file is input from a user, metadata on the audio file isextracted. In the current operation, reproduction of the audio file doesnot start.

In operation 820, video content related to the audio file is searchedfor based on the metadata extracted in operation 810.

In operation 830, if video content is found, the current processproceeds to operation 840. If video content is not found, the currentprocess proceeds to operation 860.

In operation 840, if video content has been found in operation 820, thefound video content is received. The video content may be received andreproduced through a downloading and storing process or a streamingprocess, in various embodiments.

In operation 850, the video content (video and audio components)received in operation 840 is reproduced.

In operation 860, if video content has not been found, an image relatedto the audio file is searched for. If a predetermined image stored inthe IPTV terminal is used, the current operation will be omitted.

In operation 870, the audio file is reproduced and the image found inoperation 860 is reproduced in a slide show manner at the same time.

A program for implementing a content reproduction method in the IPTVterminal according to an embodiment of the present invention can also beembodied as computer readable codes on a computer readable recordingmedium or on a computer readable transmission medium. The computerreadable recording medium is any data storage device that can store datawhich can be thereafter be read by a computer system. Examples of thecomputer readable recording medium include read-only memory (ROM),random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, andoptical data storage devices. An example of computer readabletransmission media is carrier waves (such as data transmission throughthe Internet). The computer readable recording medium can also bedistributed over network coupled computer systems so that the computerreadable code is stored and executed in a distributed fashion.

While the present invention has been particularly shown and describedwith reference to exemplary embodiments thereof, it will be understoodby those skilled in the art that various changes in form and details maybe made therein without departing from the spirit and scope of theinvention as defined by the appended claims. The exemplary embodimentsshould be considered in descriptive sense only and not for purposes oflimitation. Therefore, the scope of the invention is defined not by thedetailed description of the invention but by the appended claims, andall differences within the scope will be construed as being included inthe present invention.

What is claimed is:
 1. A content reproduction method comprising:reproducing audio content; extracting metadata associated with the audiocontent being reproduced; requesting video content corresponding to themetadata associated with the audio content being reproduced, the videocontent being requested from an external server that searches for thevideo content based on the metadata associated with the audio contentbeing reproduced; when the video content is found by the externalserver, receiving the video content found by the external server andreproducing the video content; and when the video content is not foundby the external server, reproducing a predetermined image.
 2. Thecontent reproduction method of claim 1, wherein the receiving the videocontent comprises downloading and storing the video content or streamingthe video content.
 3. The content reproduction method of claim 1,wherein the metadata is extracted from an audio file of the audiocontent or a file that is separate from the audio file.
 4. The contentreproduction method of claim 1, wherein the video content comprises atleast one of a music video, a live broadcasting program, an originalsound track video, and an advertisement.
 5. The content reproductionmethod of claim 1, wherein the reproducing the video content comprisessynchronizing a video component of the video content with the audiocontent being reproduced.
 6. The content reproduction method of claim 5,wherein the reproducing the video content comprises reproducing thesynchronized video component along with the audio content beingreproduced.
 7. The content reproduction method of claim 6, wherein thereproducing the synchronized video component comprises reproducing thevideo component from a point in time based on a current reproduction ofthe audio content being reproduced.
 8. The content reproduction methodof claim 1, wherein the reproducing the video content comprises endingthe reproducing of the audio content and reproducing the video content.9. The content reproduction method of claim 8, wherein the reproducingthe video content comprises reproducing the video content from a pointin time based on a current reproduction time of the audio content beingreproduced.
 10. The content reproduction method of claim 1, wherein thereproducing the video content comprises synchronizing the video contentwith the audio content being reproduced.
 11. The content reproductionmethod of claim 1, wherein the requesting video content corresponding tothe audio content being reproduced is performed in response to a commandto perform the reproducing of the audio content.
 12. A non-transitorycomputer readable medium comprising computer readable instructionsexecutable by a computer to perform: reproducing audio content;extracting metadata associated with the audio content being reproduced;requesting video content corresponding to the metadata associated withthe audio content being reproduced, the video content being requestedfrom an external server that searches for the video content based on themetadata associated with the audio content being reproduced; when thevideo content is found by the external server, receiving the videocontent found by the external server and reproducing the video content;and when the video content is not found by the external server,reproducing a predetermined image.
 13. The non-transitory computerreadable medium of claim 12, wherein the receiving the video contentcomprises downloading and storing the video content or streaming thevideo content.
 14. The non-transitory computer readable medium of claim12, wherein the metadata is extracted from an audio file of the audiocontent or a file that is separate from the audio file.
 15. Thenon-transitory computer readable medium of claim 12, wherein the videocontent comprises at least one of a music video, a live broadcastingprogram, an original sound track video, and an advertisement.
 16. Thenon-transitory computer readable medium of claim 12, wherein thereproducing the video content comprises synchronizing a video componentof the video content with the audio content being reproduced.
 17. Thenon-transitory computer readable medium of claim 16, wherein thereproducing the video content comprises reproducing the synchronizedvideo component along with the audio content being reproduced.
 18. Thenon-transitory computer readable medium of claim 17, wherein thereproducing the synchronized video component comprises reproducing thevideo component from a point in time based on a current reproductiontime of the audio content being reproduced.
 19. The non-transitorycomputer readable medium of claim 12, wherein the reproducing the videocontent comprises ending the reproducing of the audio content andreproducing the video content.
 20. The non-transitory computer readablemedium of claim 19, wherein the reproducing the video content furthercomprises reproducing the video content from a point in time based on acurrent reproduction of the audio content being reproduced.
 21. Thenon-transitory computer readable medium of claim 12, wherein thereproducing the video content comprises synchronizing the video contentwith the audio content being reproduced.
 22. The non-transitory computerreadable medium of claim 12, wherein the requesting video contentcorresponding to the audio content being reproduced is performed inresponse to a command to perform the reproducing of the audio content.